fix: normalize agent IDs and remove bootstrap files for benchmark#37
Conversation
Code Review SummaryStatus: 2 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Files Reviewed (1 file)
Fix these issues in Kilo Cloud Reviewed by claude-4.6-opus-20260205 · 244,628 tokens |
- Fix agent ID normalization to handle lowercase transformation - Remove BOOTSTRAP.md, SOUL.md, USER.md, IDENTITY.md before running tasks - Fix model ID normalization to preserve provider-qualified models (e.g., minimax-cn/) These fixes ensure benchmark tasks work correctly with OpenClaw agents.
zhuanghaoz
force-pushed
the
fix/agent-id-normalization
branch
from
0c5418b to
0cccb85
Compare
- Copy skills from main workspace to benchmark workspace so agents can use nano-pdf - Add 2-second delay before grading to ensure files are flushed to disk - Fix model ID normalization to preserve provider-qualified models
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
|
@zhuanghaoz thanks for this contribution I am wondering if "Remove BOOTSTRAP.md, SOUL.md, USER.md, IDENTITY.md before running tasks" is a good idea - I had problems early in this project with OpenClaw not linking not having those and getting lost on actual task completion |
|
👋 Hi @zhuanghaoz! I'm @olearycrew's OpenClaw bot doing a triage pass. This PR has merge conflicts with main that need to be resolved before it can be merged. Just flagging in case you missed it! |
Co-authored-by: kilo-code-bot[bot] <240665456+kilo-code-bot[bot]@users.noreply.github.com>
| if skill_dir_src.is_dir(): | ||
| dest_skill_dir = dest_skills_dir / skill_dir_src.name | ||
| # Copy skill directory | ||
| import shutil |
There was a problem hiding this comment.
WARNING: Redundant import shutil inside a loop — shutil is already imported at line 180 at the top of this same function. While Python caches module imports so this won't cause a runtime error, it's unnecessary and clutters the loop body. Move it to the top of the function alongside the existing import, or simply remove this line.
| import shutil | ||
| if dest_skill_dir.exists(): | ||
| shutil.rmtree(dest_skill_dir) | ||
| shutil.copytree(skill_dir_src, dest_skill_dir) |
There was a problem hiding this comment.
WARNING: shutil.copytree follows symlinks by default. If any skill directory under ~/.openclaw/workspace/skills/ contains symlinks pointing to sensitive locations (e.g., ~/.ssh, credentials), those files will be fully copied into the benchmark workspace and become accessible to the benchmark agent.
Consider using shutil.copytree(skill_dir_src, dest_skill_dir, symlinks=True) to preserve symlinks without following them, or ignore_dangling_symlinks=True at minimum. This hardens the copy against unintended data exposure in the benchmark workspace.
3 participants
These fixes ensure benchmark tasks work correctly with OpenClaw agents.